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A  method  Is  presented  for  learning  general  descriptions  of  concepts  from  • 
sequence  of  positive  and  negative  training  instances.  This  method  Involves  examining 
a  predetermined  space  or  language  of  possible  concept  descriptions,  finding  those 
which  are  consistent  with  the  observed  training  instances.  Rather  than  use  heuristic 
search  techniques  to  examine  this  concept  description  space,  the  eubspace  (version 
space)  of  all  plausible  concept  descriptions  is  represented  and  updated  with  each 
training  Instance.  This  version  space  approach  determines  all  concept  descriptions 
consistent  with  the  training  Instances,  without  backtracking  to  reexamine  past  training 
Instances  or  previously  rejected  concept  descriptions. 

The  computed  version  space  summartzes  the  information  within  the  training 
Instances  concerning  the  Identity  of  the  concept  to  be  learned.  Version  spaces  are 
therefore  useful  for  making  reliable  classifications  based  upon  partially  learned 
concepts,  and  for  proposing  informative  new  training  Instances  to  direct  further 
learning.  The  uses  of  version  spaces  for  detecting  Inconsistency  In  the  training 
Instances,  and  for  learning  in  the  presence  of  inconsistency  are  also  described. 

Proofs  are  given  for  the  correctness  of  the  method  for  representing  version 
spaces,  and  of  the  associated  concept  learning  algorithm,  for  any  countably  Infinite 
concept  description  language.  Empirical  results  obtained  from  computer 
Implementations  In  two  domains  are  presented.  The  version  space  approach  has  been 
implemented  as  one  component  of  the  Meta-DENDRAL  program  for  learning  production 
rules  In  the  domain  of  chemical  spectroscopy.  Its  implementation  in  this  program  is 
described  in  detail. 
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ordering,  have  been  defined  for  K^elflc  pettem  language,  end  used  by  [Wotkln.  "O'*  •0*e'»'o  ‘".n  the  corresponding  con.tr.lnt  In  P2. 

1S70].  [Mlchalskl,  1073],  [Hayes-floth,  1t76].  and  [Vera,  1976]  In  their  work 

Involving  inachlno  learning.  '  To  be  a  oompleta  description  of  the  partial  ordering  on  relations,  each  of  the 

three  lower  relatlona  In  the  figure  should  also  point  to  the  constraint  ‘Any  relation*. 
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'  Introducing  houristic  pruning,  howpupr.  invalidates  the  guarantee  that  clt 
concept  deacrlptiona  consistent  with  the  data  will  be  found. 
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’  I  am  Indebted  to  Lew  Creary  and  CMtoor  Srinlvasan  lot  useful  auggastlons  _ 

which  helped  to  clarify  aeveral  Issues  In  the  following  discussion.  ’  Other  criteria,  such  as  **gansraiity**  or  "elegance”,  may  also  be  Important  In 

choosing  a  pattern  to  describe  the  concept.  Hers  only  the  central  criterion  for 
acceptance,  consistency  with  the  training  Instancss,  Is  considered. 


I••fning  problem  Is  the  set  VS  of  ell  such  patterns. 
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problems  art  discussad  in  detail  .'n  chapter  6.  This  section  provides  the  theoretical 
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Formal  Troatmant 
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targat  concapt.  Tha  following  taction  contldart  tha  ralatod  problem  of  generating 


4.3  Raquasting  Naw  Training  Inttanca*  additional  inlormation  concarning  tha  target  concept.  Although  Inttences  are 


inttanu.  Thus  an  Instance  Is  chosen  whose  classification  la  axpactad  to  provide 
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concapt  daacription  may  ba  found  which  is  conaistant  with  ovary  training  Inatanca.  At 
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fragmant*  into  Infarrad  training  inatanca*  of  brokan  and  unbrokan  bond*  raault*  In  a  rules  i*  not  cufficlent  to  express  *1'  possibly  meaningful  rules  of  mass  spectroscopy. 
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many  backtracking  approaches  in  two  important  respects: 


In  order  to  deal  with  inconsistent  concept  learning  problems,  two  aeries  of 
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Instances,  and  setting  S,  to  the  set  of  maximally  specific  concept  descriptions 


Wher*  MAX(X)  and  MIN(X)  ar*  the  sets  ot  maximally  apecific  and  maximally 


patterns  consistent  with  any  4  of  the  6  observed  negative  training  instances. 


Figure  6.3  Multiple  boundary  eats  60.  61,  60,  and  61, 
oonaiatant  aat  of  training  inatancas. 
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kitrodjctlen  of  thia  training  fetalanca. 
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Involving  many  Inconsistaneias. 


Laarning  In  L«<t  P«rf act  Situations  124  Laarning  In  Less  Parfact  Situations  126 


Laarning  in  Latt  Perfect  Situations  128  Laarning  In  la«(  Parfaet  Situations  120 


Plgura  6.6  Laarnlng  a  Sisjunctiva  eoncapt. 


t««rnino  In  Ln«s  Pnrftct  Situations  130 


131  Vtrtion  Sp*c*t  and  Mata-OENORAL  132 
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Thi*  concapt  laarning  problem  may  easily  be  phrased  as  the  followlno  search 


other  program*,  auch  a*  the  RULEGEN  portion  of  tha  Mata-DCNDRAL  program  procoaalng  of  rpliabi*  training  inatanca*.  Modal-diractad  approacha*  era  wall  auitad 
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therefore  essured  to  find  the  cet  of  maximally  specific  concept  descriptions 
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past  training  Inatancas  sxpllcitly  to  aatarmlns  which  rsvitlonc  to  currant  hypothaaas 
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